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ABSTRACT 

Some  contributions  of  Information  Theory  to 
the  study  of  data  communication  networks  are  out¬ 
lined,  with  special  attention  given  to  the  problem 
of  efficient  message  addressing  and  to  the  capacitv 
of  Aloha-like  multiple  access  channels  with  infinite 
number  of  sources. 


I.  INTRODUCTION 

The  field  of  data  communication  networks  has 
been  one  of  the  most  active  topics  of  research  in 
communication  systems  during  tne  past  decade.  In¬ 
formation  Theory  had  a  modest  role  in  that  study, 
especially  when  compared  to  its  prominent  influence 
as  underlying  the  theories  of  modulation  and  coding. 
However,  besides  its  study  of  multi-user  source  and 
channel  coding.  Information  Theory  did  contribute 
to  two  challenging  aspects  of  data  networks,  in 
providing  bounds  on  the  a.T,ount  of  information  neces- 
sary  to  address  mcssage.s,  and  bounds  on  the  cfficen- 
cy  of  "Aloha- like”  multiple  access  channels. 

In  their  simplest  forms,  both  problems  are 
similar:  a  number  of  sources  can  transmit  on  a 
common  channel.  In  the  addressing  problem,  a 
server"  obser\*es  which  sources  have  messages  ready, 
and  selects  a  message  for  transmission  on  the  line. 
The  problem  is  to  decide  which  message  to  select, 
and  to  encode  efficiently  from  what  source  the  mes¬ 
sage  originates.  In  the  "Aloha-like"  multiple  ac¬ 
cess  channel,  there  is  no  central  observer.  Rather 
sources  observe  the  channel  and  decide  individuallv 
when  to  transmit.  If  only  one  source  at  a  time 
transmits,  the  message  goes  through  correctly.  If 
:two  or  more  transmit  simultaneously,  all  messages 
are  lost  and  the  sources  must  retry  later.  The 
problem  is  to  find  an  efficient  access  algorit.hm. 


I  Those  problems  e.rist  only  because  of  the  bursty 
nature  of  sources  used  in  data  networks  (they  pro¬ 
duce  short  messages  between  long  and  variable 
periods  of  silence),  and  because  it  is  required 
ithat  messages  bo  transmitted  with  short  dclavs.  If 
delays  much  longer  than  the  typical  time  between 
|thc  generation  of  messages  could  be  tolerated,  then 
almost  no  eyplicit  information  would  be  needed  for 
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the  first  problem,  and  collisions  need  not  occur 
in  the  second,  as,  by  the  ;_w  of  large  numbers, 
transmissions  could  be  sc.'-eduled  Calmest)  deterai- 
stically  after  having  buffered  a  large  number  of 
messages. 

■No  theory  has  been  found  to  deal  with  the 
questions  of  bursty  sources  and  real  time  trans¬ 
missions.  The  purpose  of  this  paper  is  to  review 
the  contribution  of  Information  Theory,  leaving 
technicalities  and  most  formulas  to  the  references, 
and  to  generate  attention  to  these  questions. 

II.  MESSAGE  ADDRESSING 

A  data  communication  networw  carries  messages 
between  a  large  number  of  users.  Wo  call  the  se¬ 
quence  of  messages  from  a  given  source  to  a  given 
destination  a  conversation.  Naturally  as  messages 
are  transmitted  in  the  network  one  must  send  not 
only  the  body  of  the  message,  but  also  information 
about  which  conversation  it  belongs  to.  Giving 
this  information  is  called  addressing,  as  often  the 
conversation  is  identified  by  the  addresses  of  its 
source  and  destination.  , 

One  way  to  address  a  message  is  to  prefi.x  it 
with  a  codeword  specifying  its  conversation,  thus 
requiring  log,  N  bits  per  message,  where  N  is  the 

nu.mber  of  conversations  using  a  link.  This  explicit 
coding  scheme  has  the  merit  of  simplicity,  but  can 
be  inefficient  when  messages  are  short  and  N  is 
Urge.  If  the  traffic  on  the  link  is  heavy  the 
inefficient  encoding  can  cause  verv  large  delays  r 
which  are  not  acceptable  in  data  networks.  j 

Another  way  of  addressing  is  to  cycle  through  ' 
the  set  of  convor.sations  using  a  link  and,  for  each 
conversation,  to  transmit  a  message  (or  part  of  a 
message)  if  one  is  waiting,  else  a  special  "empty" 
codeword.  Thus  no  explicit  address  is  transmitted, 
but  rather  the  position  of  a  message  in  a  cycle  al¬ 
lows  the  determination  of  its  conversation.  This 
scheme  works  well  for  synchronous  sources,  and  for 
bursty  sources  on  heavily  loaded  channels  where  • 
high  delays  guarantee  that  "empty"  codewords  are 
rarely  sent.  However  it  causes  relatively  high 
delays  when  traffic  is  light,  as  a  message  must  ,  !  » 

wait  for  its  turn  in  the  cycle,  thus  t,\-pically  ^  I 
times  the  time  tc  send  the  "empty"  codeword.  “  j 

:!  I  .mtvrt :  >1-  thus  apparent  that  the  addressing  scheme^ 
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■inimizing  the  expected  message  delay  depends  on 
the  characteristics  of  the  tratfic.  In  light  traf¬ 
fic  the  expected  number  of  messages  waiting  is 
small  compared  to  S,  and  explicit  coding  schemes 
are  preferred  as  the  conversation  to  which  a  mes¬ 
sage  belongs  can  hardly  be  predicted.  In  heavy 
traffic  however  the  number  of  waiting  messages  is 
large,  thus  one  can  gain  by  reordering  the  mes¬ 
sages.  so  that  the  conversation  to  which  the  next 
one  belongs  is  more  predictable  and  requires  less 
information.  In  between,  one  can  use  mixed  strat¬ 
egies.  where  one  cycles  between  groups  of  conver¬ 
sations,  distinguishing  between  conversations  in 
the  same  group  by  explicit  addresses.  The  problem 
Us  complicated  because  the  addressing  scheme  itselt 
influences  the  delay,  and  thus  the  number  of  ‘aj'* 
ing  messages.  These  concepts  are  developed  in  1). 
where  several  schemes  arc  analv:ed  and  the  rei»ult- 
Ung  queueing  delays  compared;  differences  between 
i schemes  can  be  impressive  in  heavy  traffic. 

The  above  discussion,  while  not  rigorous,  has 
the  merit  to  point  out  that  the  amount  of  address¬ 
ing  information  (in  the  information  theoretic  sense) 
depends  on  the  delay.  This  was  recognized  by  ual- 
lager  who  succeeded  in  quantifying  it  (2].  He 
noticed  that  the  arrival  times  of  the  messages  at 
the  receiver  give  information  about  the  times  the 
messages  were  generated  by  the  sources.  That  the 
receiver  is  not  interested  in  those  times  does  not 
alter  the  fact  that  the  information  is  transmitted 
and  uses  channel  capacity.  The  smaller  the  delay, 
the  more  precisely  the  arrival  time  at  the  receiver 
specifies  the  generation  time,  thus  the  more  infor¬ 
mation  is  transmitted. 

Gal  lager,  using  rate  distortion  theory'  with 
the  delay  as  distortion,  showed  that  if  the  raess.xge 
generations  for  a  conversation  arc  Poisson  with  rate 
X,  and  i^  the  expected  delay  before  delivery  at  the 
receiver  is  d,  then  the  information  supplied  to  the 
receiver  about  message  generation  times  is  at  least 
log  (l-exp(-Xd))  bits.  This  bound  is  good  for  small 
Xd.‘  For  large  Xd  Rohrs  [3]  showed  that  the  amount 

of  information  decreases  no  faster  than  l/(Xd)  , 
while  strategies  exist  wher^  it  decreases  like  j 

log  (Xd)/(Xd)-  [2).  1 

I  It  should  be  stressed  that  these  are  bounds  on 
the  amount  of  information.  This  information  must 
pf  course  be  transmitted  as  symbols.  Quite  unex¬ 
pectedly  at  first  sight  the  amount  of  ("physical'') 
bits  transmitted  is  known  exactly,  and  does  not  de¬ 
pend  on  the  actual  strategy:  C{l-p)  addressing  bits 
pre  sent  every  second  on  a  line  of  capacity  C  bits/s 
whoso  fraction  o  is  occupied  by  actual  message  bits 
It  is  so  as  all  bits  that  do  not  belong  to  the  bod¬ 
ies  of  the  messages  have  an  addressing  function. 
[This  can  be  seen  most  clearly  in  the  cyclic  scan 
mode  of  addressing  messages. 

III.  Atoiu-LIKE  MULTIPLE  ACCESS  CHANNELS 

III.l  Introduction 

Consider  the  following  model  of  the  generalized 
IaLOHA  system.  Geographically  separated,  but.,  timoj; 


synchronized  transmitters  send  and  receive  messages 
on  a  common  channel.  If  no  transmitter  is  active, 
this  fact  is  recognized  by  all  within  t^  seconds. 

If  exactly  one  transm  ttor  sends  a  .message,  the 
message  is  received  successfully  and  this  is  known 
to  all  within  t^  seconds.  Finally,  if  two  or  more 

transmitters  are  active  simultaneously,  then  a  col¬ 
lision  is  said  to  occur  and  it  is  detected  by  all 
within  t,  seconds.  All  messages  involved  in  the 

collision  must  be  retransmitted  at  a  later  time. 

This  model  represents  a  variety  of  systems. 

The  slotted  ALOHA  channel  [4]  has  'g  *  *  *2' 

Cjrrier  sense  multiple  access  radio  systems  [5]  can 
detect  idles  quickly  (carrier  not  present)  while 
they  distinguish  between  collisions  and  successes 
by  using  error  detecting  codes.  Thus  they  have 
t  «  t  «  t,.  Some  broadcast  cable  systems  (e.g., 

the  Ethernet  (o))  havea  "listen  while  transmit" 
feature  that  allows  the  quick  abortion  of  transmis¬ 
sion  when  a  collision  is  detected.  Thus  typically 
t  w  t,  «  t,.  Finally  "reservation"  systems  use 
0  2  1 

short  messages  to  reserve  time  for  longer  data  mes¬ 
sages.  The  short  messages  can  be  seen  as  an  idle/ 
collision  detection  mechanism,  and  again  t^  •  t^  « 

fl  (71. 

The  main  challenge  here  is  to  find  accessing 
schemes  with  small  expected  message  dei.xy.  _but  in 
this  paper  we  will  focus  on  a  slightly  different 
problem  and  investigate  how  average  message  delay 
behaves  when  the  number  of  sources  grows  large. 

We  define  the  utilization  of  a  channel  access 
scheme  as  the  fraction  of  the  time  during  which 
messages  are  successfully  transmitted.  We  define 
the  "capacity"  of  this  channel  as  the  supremum, 
over  all  schemes,  of  the  utilication.  If  the  number 
of  transmitters  is  finite,  then  the  capacity  is  l.^ 
Simple  schemes  like  synchronous  time  division  multi¬ 
plexing  or  round  robin  transmission  (cyclic  polling) 
avoid  collisions  and  can  achieve  this  capacity.  Un¬ 
fortunately  they  cause  relatively  long  message  de¬ 
lays  when  the  generation  rate  of  the  messages  is 
much  smaller  than  l/t^.  In  that  case  "random" 

transmission  schemes  arc  preferred.  They  allow  col¬ 
lisions  in  the  hope  of  reducing  delay.  Such  random 
schemes  are  customarily  analyzed  assuming  that  there 
are  infinitely  many  tra.ismitters ,  each  generating 
at  most  one  message  during  its  life-time,  and  that 
the  global  generation  process  of  the  messages  is 
Poisson  with  rate  X- 

■  The  capacity  of  the  channel  under  those  con¬ 
ditions  is  still  unknown.  .An  early  scheme,  the 
slotted  Aloha  [4]  strategy,  has  been  said  to  nave  an 


utilization  of  1/e  (when  t 


t,) .  but  has  been 


shown  to  be  unstable,  i.c.,  with  probability  one  its 
lUtilization  decreases  to  0  as  time  goes  by.  A  new 
class  of  protocols  has  recently  been  proposed  (S,91 
Each  of  those  has  a  maximum  utilization  X^  with  the 

property  that  the  number  of  messages  which  ave  been 
igcncratcd  but  not  yet  successfully  transmitted  will 
Wild ()«;, bounded  with  probab i  1  i ty  I  as  long  as  Xt,  <  X„.  . 


all  material  ri  this  space  will  be  oeleted 


If  At,  >  A„,  th*  utillzatioa  of  the  channel  is  A  , 

I  —  o  o 

but  the  expected  message  delay  is  infinite.  The 
largest  A^  found  to  this  day  is  .4377  [10]. 

Pippenger  first  showed  that  the  capacity  is 
boundcc  away  from  1,  in  fact  is  no  more  than  .744 
(tg'tj'tj)  [11].  lie  also  generalized  the  model  to  in¬ 
clude  channels  where  the  number  of  transmitted  mes¬ 
sages  can  be  determined  up  to  some  maximum  d,  and 
has  found  a  bound  on  the  utilization  that  is 
strictly  increasing  with  d,  converging  to  1.  .More¬ 
over  he  showed  the  existence  of  strategies  achiev¬ 
ing  utilization  arbitrarily  close  to  1  when  d  • 
Huablet  [II]  generalized  Pippenger's  results  for 
the  use  of  different  ^nd  obtained  a  bound  of 

.704  when  d«2  and  t  «  t,  •  t-.  This  is  bound  is 
oil 

sketched  below.  All  of  the  works  just  mentioned 
use  information  theoretic  concepts.  Recently  Molle 
[13]  obtained  better  results  (.673  for  fg  •  *  t,) 

by  an  elegant  and  simple  method.  A  slight  general¬ 
ization  of  his  work  is  also  presented. 

Before  proceeding  with  the  derivations  of  the 
bounds  we  will  examine  the  implication  of  these 
results.  First,  an  algorithm  that  is  efficient  for 
infinitely  many  sources  will  also  be  efficient  for 
M  <  »  sources  as  long  as  the  typical  intergeneration 
time  at  a  source  (M/A  for  symmetric  systems)  is 
longer  than  the  typical  message  delay.  In  that 
case  each  transmission  at  a  source  is  independent 
of  the  previous  one,  and  one  might  as  well  assume 
that  all  messages  have  distinct  sources. 

Secondly,  the  previously  mentioned  results 
show  the  existence  of  some  number  between  .4877 
and  .673  such  that  if  At.  is  loss  than  that  number 
the  average  message  delay  can  remain  bounded  no 
matter  the  value  of  M  (t^  *  *  *7^ •  However,  if 

'Atj  >  C,  the  average  message  delay  must  increase 

iwith  M.  It  is  readily  seen  that  the  increase  is 
I  linear  for  synchronous  time  division  multiplexing 
and  cyclic  polling. 

j  i 

|III.2  The  Information  Theoretic  Sound  ' 

I  To  understand  the  fallowing  model,  note  that 
:a  conflict  resolution  protocol  is  a  sequential  de- 
Icision  process,  thus  it  can  be  described  as  a  ter- 
jnary  tree.  Every  node  corresponds  to  an  "experi- 
|ment",  i.e.,  the  transmission  of  messages.  Branches 
|correspond  to  outcomes,  i.e.,  idle,  success,  or 
icollision.  .Associated  with  each  experiment  is  a 
[set  of  times,  typically  a  time  interval.  Only  1 

[those  messages  generated  during  the  sot  correspond- 
'ing  to  an  experiment  are  transmitted  when  the  ex¬ 
periment  is  made.  Other  conflict  resolution  algo- 
jrithms  rely  on  random  choices,  both  ways  arc  prob- 
jabilistically  equivalent  when  the  generation  times 
are  Poisson.  j 

A  protocol  for  [0,T]  is  an  infinite  ternary  ■ 
tree  in  which  there  is  an  initial  node  called  the 
root,  and  in  which  each  node  k  is  connected  by  ! 

(i)  i 

branches  to  offsprings  k^  i  «  0,1,2  that„can  ,bq 


other  nodes  or  leaves.  Every  node  k  is  labelled 
with  a  measurable  subset  y(k)  of  [O.T].  u(k)  de¬ 
notes  the  Lebesque  measure  of  ylk)  divided  by  T. 

Let  the  random  variables  E  denote  a  set  of 
Poisson  message  generation  times  in  [0,T1.  with 
expected  cardinality  u.  The  execution  of  a  proto¬ 
col  with  respect  to  a  finite  set  E  in  [O.T]  is  a 
path  through  the  tree  defined  as  follows.  Let  k  . 

o 

the  first  node  on  the  path,  be  the  root.  Suppose 

that  k  has  been  determined,  then  k  •  k^^^  where 
™  m»l  ra 

j  is  0,  1  or  2.  depending  if  the  number  of  not  yet 
transmitted  messages  in  E H  y(k  )  is  0,  1  or  more 
than  1.  ™ 

The  set  of  nodes  k  in  an  execution  I  that  have 

“Offsprings  k^  ^  is  denoted  by  S.,  the  set  of  sucess 
ful  e.xperimcnts  in  i.  *■ 

A  protocol  will  be  called  valid  if,  for  almost 
every  subset  E  of  [O.T],  the  execution  I  of  the 
protocol  with  respect  to  E  terminates  after  finite¬ 
ly  many  steps  with  ECU  y(k),  i.e.,  if  every  mes- 
kcSi 

sage  has  been  successfully  transmitted. 

The  set  of  nodes  k  in  an  execution  i  that  have 

offsprings  k^®^  or  k^*^  is  denoted  by  T-,  the  set 
of  experiments  in  i  not  resulting  in  collisions. 

A  valid  protocol  will  be  called  minimal  if  for 
all  executions  i,  y(k)  O  y(k')  »  t,  k  /  k',  k,ck*  T, 
Thus,  in  a  minimal  protocol,  a  subset  of  [0,TJ  is  * 
never  tested  again  once  it  has  been  determined  not 
to  contain  a  message,  or  when  the  only  message  pre¬ 
sent  has  been  successfully  transmitted.  Any  valid 
protocol  can  be  made  minimal  by  iteratively  chang¬ 
ing  the  ylk)'s,  starting  from  the  root,  so  as  to 
satisfy  the  null  intersection  property.  The  exe¬ 
cution  of  the  protocol  with  respect  to  a  set  E  is 
not  affected  by  the  change.  I 

1 

The  execution  of  a  protocol  is  a  random  path 
through  the  tree.  P(k)  denotes  the  probability  that 
node  k  is  included  in  an  execution,  and  q(k,i)  de¬ 
notes  the  conditional  probability  that  k^^^  follows 
k  in  the  execution  of  a  protocol. 

The  expected  number  of  experiments,  a,  in  an  i 
execution  of  a  protocol  has  value  o  «  J  P(kJ.  ' 

!  k  . 

!  The  expected  fraction  q.  of  experiments  result¬ 
ing  in  outcome  i  is  given  by^  (assuming  o<«>)  i 

q=  •  5-  y  P(k)  q(k.i)  (1) 

k 

2 

Soto  that  I  q.  •  1.  For  valid  protocols 
i-0  ^ 

qi  •  J  (2)  ^ 

We  will  denote  i’/  i- 

^  The  efficiency  e  of  a  protocol  is  simply 
1  .  _ _  i 


IB - 1!  J.'jj.  .>  lWll|8gi«yTBm 
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I 

e  a  ■  wh«r«  t.  ^  0  is  the  time  it  takes 

d  i  -  _ 

1.0  ‘ 

to  observe  outcome  i.  Note  that  for  valid  proto¬ 


cols  e  • 


''*1* 


where  f  is 


o  r 

defined  by  f  .  -  i  q.t.  and  can  be  thought  of  as 
''  ii*l  ^  " 

the  expected  time  overhead  per  message.  Note  that 
efficiencies  close  to  1  a'e  achieved  when  tj  »  f, 

which  is  t>'picall)r  the  case  for  reservation  and 
cable  systems. 

The  previous  relation  between  e  and  f  allows 
us  to  lowerbound  f  (which  does  not  depend  on  t^)  in 

order  to  upperbound  e.  IVe  will  show  that  ^  lies  in 
some  closed  convex  region  S  of  the  unit  simplex. 

The  ainimua  over  that  region  of  f  considered  as  a 
function  of  will  be  our  lower  bou.nd. 

We  first  note  that  for  any  execution  i  of  a 
oiniMl  protocol  ),  U(k)  1.  Averaging  over 

yields  i  '^^1' 

Z  P(k)W(k)(q(k.o)  ♦  q(k.l))  <  1  (3) 

k 

Next  the  entropy  h  (i.e.,  minus  the  mean  of 
the  log  of  the  probabilities)  of  the  executions  of 
a  protocol  can  be  written 


h  -  I  P(k)  H(3,(k)} 
where  H(£^(k))  «  -  > 


q(k,i)  log  q(k,i).  The  prob¬ 


ability  of  an  execution  i  of  a  minimal  protocol  is 
more  than  e  H  vu(b),  as  one  arrival  must  have 

'  bes, 

occured  in  every  y(b) ,  beSj^,  which  are  disjoint,  . 
and  no  arrival  could  have  occurred  outside  such  a 
!subset.  Thus,  h  >  E(-log  II  uu(k)e''’i* 

I  ~  keSt’  i 

^  log  e  -  y  P(k}q(k,l)logCuu(k)) .  The  right  hand 

k  I 

'side  of  the  previous  equality  is  not  less  that 

jv  log  e  -  I  P(k)q(k,l)  log  as  | 

jbe  seen  by  using  the  inequality  ln(x)  ^  x-1  and  (3)'. 
{  , 

I  Subtracting  this  last  expression  from  the  , 
'right  hand  side  of  (4) ,  dividing  by  0  and  using  (2) 

one  obtains  i  Z  P(k)  g(a(k))  >  0,  whore  g(x)  ■  * 

k  2  -  -  i 

-  X  leg  (X  **,)  -  4  X.  log(x  )  -  X  leg  e.  One: 

•  “  *  i.O  ^  ‘  ‘ 

i^l 

can  show  that  g  is  a  strictly  concave  function,  thus 


'  To  obtain  a  lowerbound  on  f,  we  find 

I 

"  1  I  » 


a  •  min  f  •  min  — -  where  S  •  (^  e  R  , 

^eS  ^eS  1 

<li  i  I  'll  ’  *•  *Ca)  10).  .  _  ‘ 

It  is  straightforward  [12]  to  find  necessary 
and  sufficient  conditions  for  optimality  and  to 
numerically  find  the  value  of  a.  The  results  are 
given  in  Figure  1. 

III. 3  A  Simpler  Bound 

The  previous  line  of  reasoning  is  interesting 
as  it  was  the  first  to  provide  a  bound  on  f,  but 
another  method,  a  slight  gcncralication  of  Molle's 
genie  argument  [13],  gives  a  better  result  while 
being  much  simpler. 

Let  us  assume  that  we  receive  the  help  of  a 
friendly  genie  which,  every  time  a  transmission 
results  in  a  collision,  pinpoints  the  two  sources 
involved  in  the  collision  that  were  the  first  to 
generate  their  messages.  The  optimal  policy  under 
those  conditions  is  obvious.  We  first  specify  a 
set  of  times  and  allow  the  transmissions  of  all  the 
messages  (e.xpected  value  denoted  by  g)  generated 
during  those  times.  If  the  outcome  is  "idle"  or 
"success"  we  just  repeat  the  cycle.  If  the  out¬ 
come  is  a  collision  we  let  the  two  souces  pointed 
to  by  the  genie  transmit  successively  before  re¬ 
peating  the  cycle.  It  is  not  desirable  to  allow 
other  sources  to  join  in  the  transmissions,  as  the 
result  would  be  equivalent  to  a  probabilistic  mix¬ 
ture  of  two  deterministic  policies;  the  one  just 
outlined  that  guarantees  success,  and  the  one  con¬ 
sisting  of  allowing  everybody  to  transmit,  which 
guarantees  that  the  genie  will  point  to  the  two 
oldest  messages  not  yet  transmitted.  This  last 
policy  gives  rise  to  a  time  overhead  per  message 

*2 

of  at  least  which  is  not  optimal  as  we  shall  see. 

For  a  given  3  the  expected  time  overhead  per 
message  is  simply  the  ratio  of  the  expected  time 
overhead  in  a  cycle  to  the  expected  number  of  mes¬ 
sages  successfully  transmitted  in  a  cycle. 

I  .  --S  .  ,,  -8  „  -8.  I 


Thus  f  >  min 


t,(l-e*®-3  e'®) 


!  3  8  e  “w  2  (l-e-“-8  e‘“)  j 

bound  is  also  plotted  in  Figure  1.  It  is  better 
than  the  previous  bound  for  small  values  of  t  /t-. 

i  0  2 

For  very  small  values  of  4^/42  approximately 

equal  to  /2~t^tJ.  j 

i 
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